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Abstract 

Background:  Approximately  70  percent  of  pediatric  care  occurs  in  ambulatory 
settings,  yet  there  has  been  little  research  on  errors  and  hann  in  these  settings. 
Given  the  importance  of  understanding  harm  in  ambulatory  pediatrics,  this  study 
was  funded  by  the  Agency  for  Healthcare  Research  and  Quality  (AHRQ)  as  part 
of  the  University  of  North  Carolina  (UNC)  Center  for  Education  and  Research  on 
Therapeutics  (CERTs),  in  partnership  with  the  American  Academy  of  Pediatrics 
(AAP)  Pediatric  Research  in  Office  Settings  (PROS)  Network.  Purpose: 

Learning  from  Errors  in  Ambulatory  Pediatrics  (LEAP)  was  designed  to  (1) 
develop  a  secure,  Web-based  tool  for  reporting  errors;  (2)  identify  the  types  and 
range  of  errors;  and  (3)  identify  errors  that  can  be  generalized  across  multiple 
practices.  Methods:  Data  collection  was  pilot-tested  in  five  pediatric  practices  in 
March  2003,  using  a  secure,  Web-based  tool.  After  revising  the  tool,  14  sites 
collected  data  from  June  to  September  2003.  Three  members  of  the  research  team 
(one  pediatrician  and  two  patient  safety  researchers)  independently  coded  the 
qualitative  error  reports  using  the  constant  comparative  method.  Reports  were 
coded  by  medical  domain,  problem  types,  and  child-specific  factors.  Coding 
discrepancies  were  reconciled  by  consensus.  Results:  Study  participants  reported 
136  errors.  Data  collection  via  the  Web-based  tool  was  very  successful; 
participating  practitioners  reported  a  high  degree  of  satisfaction  and  a  minimal 
number  of  problems.  Errors  were  reported  in  several  domains:  prevention, 
diagnosis,  treatment,  patient  identification,  communication,  falls,  equipment,  and 
administration.  For  example,  one  reported  treatment  error  was  “prescription 
changed  from  liquid  to  capsule  form  of  anticonvulsant.  Mom  misunderstood 
directions  and  gave  both  meds  for  one  week.  Child  developed  blurred  vision, 
stuttering,  and  ataxia.”  Conclusions  and  implications:  Physicians  reported 
errors,  yet  various  members  of  the  care  team  (parents,  nurses,  phannacists) 
discovered  the  errors.  This  suggests  that  everyone  has  a  role  in  preventing  errors 
from  reaching  the  child.  Information  learned  from  this  study  will  be  instrumental 
in  the  subsequent  design  of  interventions  to  reduce  errors  and  improve  pediatric 
patient  safety.  The  success  of  the  Web-based,  data  collection  tool  points  the  way 
for  future  online  data  collection  efforts.  Further  research  will  clarify  the  categories 
of  harm  observed  in  ambulatory  settings,  and  explore  venues  for  presenting  errors 
and  collaboratively  designing  and  testing  solutions. 
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Introduction 

An  estimated  70  percent  of  pediatric  care  takes  place  in  ambulatory  settings. 
This  involves  preventive,  acute,  and  chronic  illness  care.  Prior  patient  safety 
studies  in  ambulatory  and  in-patient  adult  populations  have  demonstrated  that 
errors  are  common,  costly,  and  often  serious  or  fatal.  ’  It  seems  likely  that  this 
would  also  be  the  case  with  errors  in  the  pediatric  outpatient  setting,  but 
researchers  are  faced  with  a  significant  lack  of  published  research  on  the  nature, 
frequency,  and  causes  of  ambulatory  pediatric  errors. 

Even  with  the  lack  of  data  documenting  error  and  hann  in  ambulatory 
pediatrics,  there  is  reason  to  believe  that  providing  safe  care  to  children  poses 
unique  challenges.  Because  of  their  limited  ability  to  communicate,  children  may 
be  unable  to  indicate  whether  they  are  experiencing  adverse  events,  and  those 
events  may  go  unnoticed  until  serious  harm  results.  Medications  are  a  special 
source  of  concern  for  pediatric  patients.  The  specific  risks  described  in  the 
ambulatory  setting  include  the  need  for  rapid  dose  calculation  as  well  as  parental 
administration,  which  depends  on  clear  communication  between  the  clinician  and 
the  parent.  Additionally,  the  need  for  weight-based  dosing,  when  the  patient’s 
weight  may  not  be  known  precisely,  increases  the  risk  of  either  under-  or 
overdosing  a  pediatric  patient.  Pharmacists  must  often  dilute  stock  medications 
before  they  can  be  administered  to  pediatric  patients,  providing  opportunities  for 
errors  that  can  lead  to  a  range  of  outcomes  in  the  inpatient  environment,  ranging 
from  no  hann  to  temporary  or  permanent  disability  to  death.  Because  of  the  age  of 
pediatric  patients,  when  an  error  leads  to  injury  it  may  have  an  impact  on  the 
development  and  life  course  of  the  child. 

A  study  conducted  by  the  American  Academy  of  Family  Physicians  (AAFP) 
confirms  that  enors  occur  in  the  outpatient  care  of  children.  Of  the  errors 
observed  by  family  physicians  in  their  own  practice  settings,  17.5  percent 
occurred  during  the  treatment  of  children  under  the  age  of  14  years.  Avoidable 
errors  in  ambulatory  settings  may  often  appear  to  be  trivial,  but  the  study  found 
that  approximately  5  percent  of  errors  directly  precipitated  a  hospital  admission. 
The  researchers  suggested  error  reduction  strategies  aimed  at  the  system  of  care 
delivery,  such  as  improved  chart  management,  more  effective  communication 
within  the  primary  care  team,  and  more  effective  communication  between  the 
primary  care  provider  and  other  health  care  providers. 

The  Institute  of  Medicine  report,  To  Err  Is  Human:  Building  a  Safer  Health 
System,  suggests  that  medication  errors  are  the  most  frequent  type  of  adverse 
events.  In  one  of  the  few  studies  documenting  the  epidemiology  of  medication 
errors  in  a  children’s  hospital  setting,  Kaushal  and  colleagues5' 6  found  that  serious 
pediatric  medication  errors  (potential  adverse  drug  events  and  preventable  adverse 
drug  events)  occurred  at  a  three-fold  higher  rate  in  children  than  in  adults.  This 
study  provided  important  confirmation  of  the  unique  epidemiology  of  medication 
errors  in  pediatric  inpatient  populations  and  suggests  that  hospitalized  children  are 
at  a  greater  risk  of  serious  medication  errors  than  adults. 
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More  recently,  articles  in  the  January/February  2004  issue  of  Ambulatory 
Pediatrics  described  the  state  of  existing  patient  safety  knowledge  in  pediatric 
ambulatory  care.6  8  This  group  of  articles  focused  on  general  pediatric  ambulatory 
care  literature,  the  role  of  infonnation  technology  in  improving  pediatric  patient 
safety,  the  patient  safety  risks  in  the  Emergency  Department,  and  the  state  of  our 
knowledge  regarding  medication  errors  in  pediatrics.  While  the  authors  did  make 
inferences  about  particular  risks  based  on  the  characteristics  of  children  and  the 
features  of  the  ambulatory  context,  these  articles  underscored  the  limited  research 
that  is  focused  on  the  ambulatory  care  environment  and,  in  particular,  on  pediatric 
ambulatory  care. 

To  begin  an  investigation  of  errors  in  ambulatory  pediatric  practice,  a  focus 
group  with  approximately  50  pediatricians  who  were  chapter  coordinators  (State 
representatives)  for  the  practice-based  research  network  of  the  American 
Academy  of  Pediatrics — Pediatric  Research  in  Office  Settings  (PROS) — was 
conducted  in  October  2000.  During  the  focus  group  session,  many  pediatricians 
acknowledged  that  errors  with  the  potential  to  harm  patients  happened  in  their 
primary  care  practices.  They  felt  that  they  underestimate  the  prevalence  of  errors 
in  ambulatory  settings,  in  part,  because  they  often  do  not  see  the  consequences  of 
the  errors  because  those  consequences  occur  after  the  child  leaves  the  office.  They 
commonly  mentioned  medication  errors,  including  prescribing  and  administering 
the  wrong  dosages;  prescribing  medication  despite  a  known  allergy;  giving 
inaccurate  telephone  advice  about  dosage;  and  using  the  wrong  dosage  or  wrong 
technique  to  give  immunizations.  Furthennore,  these  providers  acknowledged  that 
infants  and  children  are  likely  to  depend  on  one  or  more  adult  caregivers  to 
administer  medications,  which  also  may  increase  the  likelihood  of  an  error. 
Finally,  the  pediatricians  said  they  did  not  have  a  mechanism  for  documenting  or 
reporting  errors  in  their  practice  settings. 

Study  objectives 

In  2001,  the  Agency  for  Healthcare  Research  and  Quality  funded  the 
University  of  North  Carolina  (UNC)  Center  for  Education  and  Research  on 
Therapeutics  (CERTs)  to  conduct  a  study  specifically  designed  to  develop  a 
secure  Web-based  reporting  tool  for  collecting  data  on  errors  in  ambulatory 
pediatric  settings,  and  to  identify  the  types  of  errors  that  can  be  generalized  across 
multiple  practices.  UNC  CERTs  partnered  with  the  PROS  Network  to  conduct  a 
research  study,  called  the  Learning  from  Errors  in  Ambulatory  Pediatrics  (LEAP) 
Study,  to  better  understand  the  range  and  types  of  errors  common  in  ambulatory 
pediatric  settings.  The  research  team  developed  a  protocol  for  documenting  errors 
that  (1)  created  a  secure,  Web-based  tool  for  reporting  errors;  (2)  identified  the 
type  and  range  of  errors;  and  (3)  identified  errors  that  can  be  generalized  across 
multiple  practices.  This  article  describes  the  LEAP  Study,  presents  the  study 
results,  discusses  the  conclusions,  and  considers  potential  directions  for  future 
research. 

PROS  was  established  by  the  American  Academy  of  Pediatrics  (AAP)  in 
1986. 9  As  of  January  2004,  PROS  consisted  of  1,938  pediatric  practitioners  from 
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701  practices  in  50  States,  Puerto  Rico,  and  Canada,  teamed  with  a  research  staff 
at  AAP  headquarters  in  Elk  Grove  Village,  IL,  and  research  consultants  from 
around  the  country.  PROS-affiliated  practitioners  care  for  more  than  2  million 
children.  The  mission  of  PROS  is  to  improve  the  health  of  children  by  conducting 
collaborative  practice-based  research  to  enhance  primary  care  practice.  PROS  has 
studied  child  health  topics  as  diverse  as  the  prevalence  of  preschool  vision 
screening,10  the  onset  of  secondary  sexual  characteristics  in  young  girls,11  the 
immunization  status  of  children  seen  in  private  practice,  “  and  the  management  of 
pediatric  patients  with  psychosocial  problems  by  primary  care  providers.13  The 
LEAP  study  was  the  first  study  of  medical  errors  in  a  pediatric  practice-based 
research  network. 


Methods 

The  reporting  tool  developed  for  this  project  is  shown  in  Appendix  A.* 
Because  this  study  was  intended  to  be  an  exploratory  look  at  errors  and  “near 
misses,”  the  reporting  tool  included  several  open-ended  questions.  The  reporting 
tool  was  developed  based  on  initial  conversations  with  the  AAFP,  and  then 
revised  and  adapted  to  meet  the  needs  of  the  PROS  practitioners.  The  structure  of 
the  PROS  Network  provided  an  opportunity  for  chapter  coordinators  to  discuss 
the  reporting  tool  fonnat  and  questions  during  their  bi-annual  PROS  meetings. 
These  meetings  also  provided  an  opportunity  to  get  buy-in  from  participants  well 
before  the  data  collection  period  began. 

We  used  error  reporting  as  a  tool  to  help  us  obtain  “contextual  data”  about 
errors.  For  the  purposes  of  this  study,  errors  were  defined  as  “the  failure  of  a 
planned  action  to  be  completed  as  intended,  or  the  use  of  a  wrong  plan  to  achieve 
an  aim;  including  problems  in  practice,  products,  procedures,  and  systems.”14  The 
reporting  tool  was  developed  as  an  Internet-based,  digitally  encrypted  tool. 

Immediately  following  funding  and  Institutional  Review  Board  (IRB) 
approval,  PROS  staff  recruited  five  pediatricians  from  the  network  to  participate 
in  pilot-testing  the  Web-based  reporting  tool.  Each  of  these  pediatricians 
participated  in  a  15-minute  training  session  via  telephone.  In  addition,  they 
received  a  detailed  instruction  manual  that  explained  the  study  protocol  and  how 
to  access  and  use  the  Web-based  tool.  Pediatricians  were  then  asked  to  use  the 
tool  to  report  all  errors  that  occurred  in  a  1 -month  period.  Participants  were  asked 
to  report  each  error  that  occurred  during  the  study  data  collection  period  that  he  or 
she  was  either  directly  involved  in  or  discovered  while  caring  for  patients  in  an 
ambulatory  pediatric  setting. 

Furthermore,  practitioners  were  instructed  to  not  include  any  information  that 
might  identify  the  practitioner,  the  practice,  office  staff,  another  practitioner,  or 
the  patient.  This  was  verified  by  a  staff  person  at  PROS  who  de-identified  any 


The  appendix  cited  in  this  report  is  provided  electronically  at  http://www.aap.org/pros/ 
leapmain.htm. 
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data  that  might  have  been  inadvertently  included  before  the  reports  were  passed  to 
the  principal  investigators. 

Data  collection  was  pilot-tested  in  five  pediatric  practices  in  March  2003, 
using  the  secure,  Web-based  tool.  During  the  1 -month  pilot  test,  108  errors  were 
reported.  Review  of  the  reported  errors  revealed  that  numerous  reports  were  made 
for  “trivial”  errors  (e.g.,  inconsequential  misspellings  or  immediately  corrected 
clinical  documentation  that  had  no  reasonably  foreseeable  potential  to  cause 
hann).  The  directions  to  practitioners  about  the  reporting  tool  were  revised  to 
instruct  practitioners  not  to  the  report  minor,  nonsubstantive  errors  that  often 
occur.  After  revising  the  tool  and  the  error-reporting  instructions,  an  additional  10 
sites  were  recruited  (one  practitioner  was  not  able  to  participate  in  the  second 
round)  and  a  total  of  14  sites  collected  data  from  June  to  September  2003.  This  3- 
month  data  collection  period  resulted  in  64  error  reports. 

The  research  team  decided  to  combine  the  data  from  the  1 -month  pilot  test 
(n  =  108)  and  the  3-month  reporting  phase  (n  =  64)  in  order  to  have  more  reports 
(n  =  172)  to  analyze.  Data  were  cleaned  to  eliminate  “trivial”  reports;  this 
eliminated  44  reports  (42  of  which  were  from  the  pilot  data,  prior  to  clarifying  the 
definition  of  reportable  errors).  Error  reports  were  then  read  to  ensure  that  each 
report  represented  a  mutually  exclusive  error.  Reports  that  included  more  than  one 
error  were  divided  into  separate  error  reports.  Reading  through  the  error  reports 
and  dividing  them  into  mutually  exclusive  errors  resulted  in  19  additional  error 
reports,  for  a  total  of  147  reports.  For  example,  it  was  decided  that  this  report 
included  two  separate  errors: 

Parent  left  message  for  refill  of  Adderall  15mg,  lQDay.  Nurse 
wrote  message  for  Adderall  XR  15  mg,  lQDay.  Mom  discovered 
change  at  pharmacy  and  didn  ’t  want  Adderall  XR  because  of  high 
cost.  Upon  further  questioning,  [clinicians]  learned  patien  t  was 
actually  taking  V2  pill  in  morning  and  V2  at  lunch. 

The  Critical  Incident  Analysis  Technique15 — a  method  first  employed  in 
aviation  to  examine  aircraft  training  accidents  and  now  frequently  being  applied 
to  medicine — was  the  method  used  to  analyze  the  error  reports.16  18  It  is  a  method 
for  organizing  reports  of  critical  incidents  into  meaningful  categories  of  types.  In 
this  investigation,  the  aim  of  the  study  was  to  identify  the  types  of  errors  that 
occur  in  pediatric  ambulatory  care  and  the  aspects  of  ambulatory  medical  care  in 
which  they  occur.  The  physician  reports  provided  the  incidents  for  analysis. 

The  Constant  Comparative  Method19  provided  the  analytic  structure  for  the 
Critical  Incident  Analysis.  This  involves  reviewing  all  the  error  reports,  focusing 
on  several  reports  to  begin  to  fonnulate  meaning,  and  then  incrementally 
including  more  reports  to  begin  to  develop  preliminary  categories.  Reports  were 
coded  into  the  preliminary  categories  to  determine  the  need  to  add  new  categories, 
and  related  categories  were  grouped  to  develop  overarching  categories.  The 
typology  of  categories  was  reviewed  deductively  to  ensure  coherence  and 
parallelism.  Finally,  all  of  the  data  were  coded  using  the  developed  categories.19 
Reliability  of  the  categories  was  tested  through  a  process  of  triangulation,  which 
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involved  independent  coding  of  the  error  reports  by  three  independent  reviewers 
(one  pediatrician  and  two  patient  safety  researchers). 


This  process  of  independent  coding,  triangulation,  and  reliability  testing  was 
performed  on  two  different  sets  of  categories:  the  aspect  of  ambulatory  medical 
care  in  which  the  problems  occurred  (medical  domains:  Table  1)  and  the  types  of 
problems  reported  (problem  types:  Table  2).  Coding  discrepancies  were 
reconciled  by  consensus  of  the  research  team.  The  intercoder  reliability  for 
medical  domains  was  87  percent  and  for  problem  types,  the  intercoder  reliability 
was  72  percent. 


Table  1.  Medical  domains  used  for  coding  error  reports 


Domain 

Definition 

Preventive  medicine 

Immunization  and  age-appropriate  preventive  screenings 

Diagnostic 

Diagnostics  based  on  presenting  symptoms,  including 
selection  and  performance  of  appropriate  diagnostic  tests  and 
interpretation  of  test  results 

Medical  treatment 

Activities  within  medical  treatment,  including  the  medication 
process,  surgical/non-surgical  procedures, 
consultations/referrals  with  other  providers 

Medication 

The  medication  process,  including  ordering,  transcribing, 
dispensing,  administering 

Surgical/nonsurgical 

procedure 

A  procedure,  either  surgical  or  nonsurgical,  done  within  the 
office  setting 

Appointments/follow-up 
communication  with 
other  providers 

Activities  involved  in  ordering  and  scheduling  follow-up  visit, 
referrals,  or  consultations 

Other  medical  treatment 

Other  medical  treatment  activities  that  do  not  fit  within  the 
domains  of  medication,  surgical/non-surgical  procedure, 
communication  with  other  providers  (e.g.,  mental  health) 

Patient  identification 

Correctly  identifying  the  patient  and  the  records  that  belong  to 
the  patient 

Communication  to  patient 

Communication  from  the  practitioner  to  the  patient  regarding 
care — preventive,  diagnostic,  medication  related,  surgically 
related,  aftercare  related,  other  medical  treatment  related 

Falls 

Patient  falls  within  the  clinic  setting 

Equipment 

Equipment  maintenance,  functioning,  availability,  and 
appropriate  use 

Administrative 

Chart-related  activities,  other  clinically  significant  administrative 
activities,  or  billing  activities 
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Table  2.  Problem  types  used  for  coding  error  reports 


Problem  Type 

Definition 

Problematic 
communication  and/or 
handoffs 

Missing  or  wrong  information  or  misinterpretation  or  information 
provided  inappropriately 

Problematic  decision 

Inaccurate  decision  or  decision  that  unnecessarily  increased 
risk 

Problematic  execution 

Wrong  execution  or  failure  to  complete  an  intended  action 

Equipment 

Equipment  maintenance,  functioning,  availability,  and 
appropriate  use 

Results 

Clinicians  familiar  with  participating  in  research  efforts  of  primary  care 
networks  successfully  used  a  Web-based  reporting  system  to  document  practice 
errors.  In  a  brief  post-study  survey,  the  clinicians  expressed  a  very  high  degree  of 
satisfaction  with  the  ease  of  using  the  online  reporting  tool. 

Tables  3  and  4  provide  the  categories  of  medical  error  and  the  frequency  of 
reports.  Errors  in  the  medical  treatment  domain  were  most  frequent  (38  percent) 
followed  by  administrative  errors  (22  percent).  Reported  errors  occurred  most 
commonly  in  the  domains  of  preventive  screening  and  immunization  and  of 
medical  treatment  (specifically,  the  medication  process).  Of  the  errors  within  the 
medical  treatment  domain  (Table  4),  84  percent  were  medication  errors. 
Communication  problems  were  the  most  common  cause  of  the  reported  errors, 
with  problematic  communication  reported  as  contributing  to  67  percent  of  the 
errors.  Table  5  provides  examples,  from  the  data,  of  error  reports  coded  by 
medical  domain  and  problem  type. 


Table  3.  Categories  of  medical  error  and  frequency  of  harm 


Medical  domain  categories 

No.  of  errors 
(%  total) 

Preventive  medicine 

22 

(15) 

Diagnostic 

19 

(13) 

Medical  treatment 

56 

(38) 

Patient  identification 

4 

(3) 

Communication  to  patient 

12 

(8) 

Falls 

1 

(1) 

Equipment 

1 

(1) 

Administrative 

32 

(22) 

Total  for  medical  domains 

147 

(100) 
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Table  3.  Categories  of  medical  error  and  frequency  of  harm,  cont. 


Problem  type  categories 

No.  of  errors 
(%  total) 

Problematic  communication 
and/or  handoffs 

99 

(67) 

Problematic  decision 

14 

(10) 

Problematic  execution 

27 

(18) 

Mechanical/technical  malfunction 

6 

(4) 

Other 

1 

(1) 

Total  for  problem  types 

147 

(100) 

Table  4.  Types  of  errors  within  the  medical  treatment  category  and  frequency  of  harm 


Medical  Treatment  Categories 

No.  of  errors 
(%  total) 

Medication 

47 

(84) 

Failure  to  order 

14 

(30) 

Ordering 

26 

(55) 

Transcribing 

1 

(2) 

Dispensing 

1 

(2) 

Administration 

5 

(11) 

Surgical/nonsurgical  procedures 

2 

(4) 

Appointments/follow-up 
communication  with  other 
providers 

6 

(11) 

Other  medical  treatment 

1 

(2) 

Total  medical  treatment  domains 

56 

(100) 

362 


Pediatric  Outpatient  Errors 


Table  5.  Examples  of  error  reports 


Medical  domain 
categories 

Example  error  report 

Preventive  Medicine 

“At  4  month  visit  the  baby  received  a  Pediarix  vaccine  [a  five-in-one 
vaccine  that  combines  diphtheria,  tetanus,  pertussis,  hepatitis  B,  and 
polio]  giving  her  third  Hepatitis  B  two  months  too  early.”  (Problem  type: 
Execution) 

“Developmental  delay  in  speech  was  not  noted  at  previous  checkups.” 
(Problem  type:  Decision) 

Diagnostic 

“Patient  with  E.coli  oh  157.  Needed  follow-up  stools  to  get  back  to  day 
care.  Order  faxed  to  lab.  Lab  processed  stool  based  on  prior  stool 
order  for  rotavirus/O  and  IP  from  2  wks  prior  rather  than  culture. 

Delayed  return  to  day  care.”  (Problem  type:  Execution) 

“After  discussing  with  an  ER  doc  a  19-day-old  patient  being  admitted 
to  the  hospital  for  pneumonia,  primary  care  doc  ordered  cefuroxime 
and  gentamicin  over  the  phone  because  he  was  tied  up  in  the  office 
for  the  next  5  hours.  After  examining  the  patient,  looking  at  the  x-ray, 
and  reviewing  lab  results,  it  was  apparent  that  the  child  did  not  have 
pneumonia  but  a  fever  and  a  neurological  process.”  (Problem  type: 
Decision) 

Medical  treatment 

Medication 

“Impetigo  diagnosed  during  well  visit,  and  forgot  to  give  Rx  at  end  of 
visit.”  (Problem  type:  Communication) 

“6-year-old  boy  on  5  different  medications  prescribed  by  psychiatrist 
for  behavior  disorder.  1  prescribed  an  antibiotic  for  bronchitis  with 
wheezing  that  reacts  with  one  of  his  psychiatric  meds.”  (Problem  type: 
Decision) 

“Parent  left  message  for  refill  of  Adderall  15mg,  IQday.  Upon  further 
questioning,  learned  pt  was  actually  taking  1/2  pill  in  morning  and  1/2 
pill  at  lunch.”  (Problem  type:  Communication) 

“Mom  given  written  instructions  for  psychotropic  med  —  Adderall.  Told 
to  give  1/2  tab  BID  and  interpreted  it  as  bedtime.”  (Problem  type: 
Communication) 

Surgical/non-surgical 

procedure 

“While  doing  a  circumcision  with  a  Gomco  device,  bleeding  was  noted, 
clamp  had  become  dislodged.”  (Problem  type:  Execution) 

“1  forgot  to  put  gloves  on  until  1  had  started  a  circumcision.”  (Problem 
type:  Execution) 

Appointments/follow¬ 
up  communication 
with  other  providers 

“Infant  showed  up  in  office  with  significant  distress  after  cardiac 
surgery.  Cardiologist  and  cardiothoracic  surgeons  had  not 
communicated  back  to  our  office,  and  they  were  not  available  by 
phone  to  comanage  the  patient.  Parents  of  patient  had  not  been  given 
good  instructions  for  follow  up  after  surgery  if  child  became  ill.” 

(Problem  type:  Communication) 

“On  well  visit,  had  borderline  vision  screening  and  forgot  to  refer  to 
optometry  at  end  of  visit.”  (Problem  type:  Communication) 

Other  medical 
treatment 

“1  circled  breathing  treatment  instead  of  tympanometry.  The  nurse 
asked  me  what  dosage  1  wanted  for  the  breathing  treatment,  since  1 
hadn’t  written  one.”  (Problem  type:  Communication) 
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Table  5.  Examples  of  error  reports,  cont. 


Medical  domain 
categories 

Example  error  report 

Patient  identification 

“Information  on  the  wrong  patient  with  the  same  name  in  this  patient’s 
chart,  regarding  allergies  and  medications.  When  in  for  a  physical,  the 
clinician  initiated  a  discussion  of  allergies  as  though  it  was  the  other 
patient.  Error  discovered.  There  was  a  warning  message  written  on 
the  outside  of  the  chart  RE  two  patients  with  same  name,  but  this 
didn’t  prevent.”  (Problem  type:  Communication) 

Communication  to 
patient 

“Pediatric  Neurologist  wanted  to  change  the  patient  from  liquid  to 
capsule  form  of  anticonvulsant.  Mom  misunderstood  the  directions  and 
gave  both  meds  for  a  week.  Child  developed  blurred  vision,  stuttering 
and  ataxia.”  (Problem  type:  Communication) 

Falls 

“Pt.  got  tetanus  shot.  Claimed  to  be  OK  after.  Was  in  elevator  leaving 
office  and  fainted.  No  complete  loss  of  consciousness  -  mom  caught 
him  with  no  head  injury  sustained.  Brought  back  to  office  where  put  on 
couch  until  color  back.”  (Problem  type:  Other) 

Equipment 

“Dictation  was  not  completed  due  to  a  new  technical  problem  not 
encountered  with  this  dictation  equipment  before.”  (Problem  type: 
Mechanical/Technical  Malfunction) 

Administrative 

“The  wrong  appointment  date  was  put  in  the  computer  for  a  patient. 
However  she  also  was  scheduled  for  the  right  date  (in  3  weeks).  We 
called  to  check  today  since  she  didn’t  show,  and  found  that  the  parent 
wasn’t  aware  of  today’s  appt  but  did  have  the  right  appt  given  to  her 
on  a  card.  1  don’t  know  how  this  wrong  date  got  in  the  computer.” 
(Problem  type:  Execution) 

Problem  Type 
Categories 

Example  error  report 

Problematic 
communication  and/or 
hand  offs 

“Prescription  refill  written  for  Adderall  5  mg  #30  because  note  from 
office  personnel  did  not  indicate  that  child  usually  receives  #60. 

Because  1  rarely  write  for  #60  Adderall,  1  wrote  for  the  smaller 
number.”  (Medical  domain:  Medication  Ordering) 

Problematic  decision 

“Ordered  the  wrong  immunization  for  patient  -  IPV  instead  of  PCV.” 
(Medical  Domain:  Preventive  Medicine) 

“1  prescribed  a  medication  that  was  questionably  an  allergy  risk.  The 
regular  caretaker  didn’t  bring  him  in,  so  1  had  less  info  than  usual  to  go 
on.”  (Medical  Domain:  Communication  to  patient/family) 

Problematic  execution 

“Child  missed  getting  an  ordered  vaccine.”  (Medical  domain: 

Preventive  medicine) 

Mechanical/technical 

malfunction 

“A  low  gestational  age  post-term  newborn  infant  with  mild  respiratory 
distress  syndrome  requiring  continuous  positive  airway  pressure  only 
on  room  air  got  100  percent  02  for  four  hours  due  to  some  difficulty 
with  the  respiratory  equipment.”  (Medical  domain:  Medication 
Administration) 

Physicians  reported  errors,  yet  various  members  of  the  care  team  (parents, 
nurses,  pharmacists)  often  discovered  the  errors.  As  we  expected,  physicians 
discovered  most  of  the  errors  (52  percent).  Parents  and  nurses  discovered  15 
percent  and  13  percent,  respectively.  Other  errors  were  discovered  by 
transcriptionists  (7  percent),  pharmacists  (4  percent),  lab  technicians  (3  percent), 
and  other  physicians  (1  percent).  Office  receptionists,  school  personnel,  a 
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grandmother,  and  a  patient  found  the  remaining  3  percent  of  the  errors.  While  we 
thought  the  person  reporting  the  error  (the  pediatrician)  would  be  the  one  who 
most  often  discovered  the  error,  these  results  suggest  the  important  role  that  the 
entire  care  team  has  in  preventing  errors  from  reaching  the  child. 

The  occurrence  of  an  error  did  not  imply  harm  or  injury  to  the  patient.  When 
practitioners  were  asked  to  report  whether  an  error  resulted  in  harm,  they  reported 
that  6  percent  (n  =  9)  of  the  errors  resulted  in  harm,  and  five  percent  (n  =  7) 
scored  as  “don’t  know.”  The  prospective  nature  of  this  study  allowed  us  to  detect 
errors  that  did  not  result  in  harm  to  the  patient — these  types  of  errors  would  be 
difficult  to  identify  in  a  retrospective  chart  review. 

Anecdotally,  physicians  described  their  personal  learning  about  identifying 
errors  and  noted  an  improvement  in  the  culture  of  learning  in  their  practices.  For 
example,  one  pediatrician  said,  “At  first  I  was  embarrassed  to  let  other  people  in 
my  office  know  that  I  was  part  of  this  study  and  that  I  made  errors.  But  by  the  end 
[of  the  reporting  period],  I  was  going  up  to  people,  saying  ‘let  me  tell  you  about 
an  error  I  made — has  this  ever  happened  to  you?’”  This  sort  of  change,  and 
development  of  “a  culture  of  learning,”  is  an  important  step  toward  developing  a 
safer  health  care  environment. 

Participating  clinicians  also  reported  anecdotally  an  increased  awareness  of 
potential  errors  and  harm  in  practice.  One  clinician  mentioned  the  tendency  to 
gloss  over  minor,  inconsequential  errors,  especially  during  very  busy  times.  She 
noted  that  study  participation  raised  her  consciousness  about  errors,  and 
encouraged  her  to  mention  minor  errors  that  would  otherwise  go  unnoticed,  to 
practice  colleagues. 

Another  clinician  reported  that  the  busy  nature  of  modem  medicine,  in  itself, 
might  contribute  to  medical  errors  and  to  the  glossing  over  of  minor  medical 
errors,  due  to  the  clinician  being  too  busy  to  notice  or  take  action.  This  is  another 
example  of  how  the  development  of  a  “culture  of  learning”  might  lead  to  a  safer 
health  care  environment.  Several  clinicians  reported  a  fair  degree  of  subjectivity 
over  the  definition  of  an  error,  the  degree  of  its  seriousness,  and  whether  to  report 
it.  Another  important  facet  of  raising  clinicians’  awareness  about  errors  in  their 
practice — and  of  reporting  medical  error  data  in  general — is  a  clearer  definition 
about  what  constitutes  an  error,  what  leads  to  an  error,  and  what  can  be  done  to 
prevent  it. 

Limitations 

Several  limitations  require  clarification.  This  study  represents  a  first  pass  at 
developing  a  convenient  and  efficient  reporting  system  for  collecting  medical 
errors  in  ambulatory  pediatric  practices.  Because  of  the  pilot  nature  of  this  study, 
only  a  small  number  of  practitioners  within  the  PROS  Network  were  able  to 
participate  in  the  project.  Therefore,  the  data  collected  are  not  necessarily 
representative  of  the  entire  PROS  Network,  but  rather  offer  an  “initial  peek.”  In 
addition,  only  one  person  (a  pediatrician)  from  each  practice  reported  errors. 
Furthermore,  the  fact  that  the  person  reporting  the  error  was  also  directly 
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responsible  for  the  clinical  care  of  the  patient  may  have  biased  the  errors  that 
were,  and  were  not,  reported.  Also,  the  PROS  clinicians  are  familiar  with 
participating  in  primary  care  network  research  efforts — this  may  have  influenced 
their  willingness  to  participate. 

It  is  a  widely  held  belief  that  medical  errors  are  a  significant  problem,  yet  it  is 
difficult  to  estimate  the  actual  incidence  of  error.  This  study  did  not  attempt  to 
identify  an  error  rate.  Although  we  encouraged  practitioners  to  report  all  errors,  it 
is  likely  that  some  errors  were  undetected,  and  others  were  detected  but  were 
unreported.  One  clinician  reported  noticing  errors  by  colleagues  who  either  didn’t 
notice  them  or  didn’t  think  they  were  important  because  there  was  not  a  negative 
consequence. 

While  this  study  cannot  provide  incidence  rates  of  errors,  the  reports  establish 
the  occurrence  of  that  error  type  and  provide  a  rich,  descriptive  database  of  a 
variety  of  errors. 

An  additional  limitation  of  our  reporting  tool  is  that  it  didn’t  collect  data  on 
categories  of  harm.  Instead,  it  just  asked  practitioners  to  report  whether  the  error 
resulted  in  any  harm.  Obviously,  not  all  medical  errors  result  in  harm  to  the 
patient,  yet  most  have  the  potential  to  cause  harm. 


Discussion 

The  LEAP  study  confirms  that  medical  errors  occur  in  ambulatory  pediatric 
settings.  LEAP  demonstrates  that  data  collection  of  medical  error  reports  can  be 
successfully  accomplished  in  busy  ambulatory  practice  settings  via  a  Web-based 
tool.  In  addition,  LEAP  confirms  what  other  studies  have  documented:  medication 
errors  are  among  the  most  common  challenges  in  patient  safety.  Participating 
practitioners  reported  a  high  degree  of  satisfaction  and  a  minimal  number  of 
problems  with  the  Web-based  tool.  The  success  of  this  Web-based  data  collection 
tool  points  the  way  for  future  online  data  collection  efforts. 

Although  error  reporting  was  a  component  of  this  study,  it  is  not  the  end-goal. 
Making  care  safer  for  children  is  our  ultimate  objective.  LEAP  suggests  the  utility 
of  a  simple  mechanism  for  collecting  and  synthesizing  errors.  Anecdotal  reports 
from  participating  clinicians  indicate  that  this  process  facilitated  important 
reflection  about  safety  issues  in  outpatient  pediatric  health  care  settings. 
Identifying  and  learning  from  “near-misses”  or  errors  that  do  not  result  in  hann  is 
the  initial  step  in  making  health  care  safer.  Training  in  error  identification  and 
error  mechanisms,  and  in  the  systems,  culture,  conditions,  and  practices  of  patient 
safety  can  provide  the  necessary  tools  for  clinicians  and  office-based  practice  staff 
to  learn  about  key  safety  issues  of  communication,  hand  offs,  and  teamwork. 
Working  collaboratively  to  test  solutions  may  lead  to  the  redesign  of  practice 
systems  to  improve  the  safety  of  the  pediatric  medical  care  delivered  in  their 
office. 
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Conclusion 

Information  obtained  from  this  study  will  be  instrumental  in  the  design  of 
interventions  to  reduce  errors  and  improve  pediatric  patient  safety  in  the 
outpatient  setting.  The  LEAP  study  documents  that  errors  occur  in  ambulatory 
settings,  most  frequently  in  the  context  of  medical  treatment.  Pediatricians 
successfully  used  a  Web-based  reporting  tool  over  a  short  period  of  time. 

Reported  errors  occurred  most  frequently  in  the  domains  of  preventive  screening 
and  immunization,  and  of  medical  treatment  (specifically  the  medication  process). 
Communication  problems  were  the  most  common  cause  of  the  reported  errors. 

Our  results  suggest  that  all  members  of  the  health  care  team  play  a  role  in 
preventing  errors  from  reaching  the  child.  Further  research  will  clarify  categories 
of  harm  in  ambulatory  settings  and  explore  venues  for  presenting  errors  and 
collaboratively  designing  and  testing  solutions.  For  future  research,  we  will 
consider  a  larger  study  that  will  include  a  more  representative  sample  of  practices, 
as  well  as  the  participation  by  additional  practice  staff  and  possibly  parents. 

Improvement  in  ambulatory  care  safety  will  continue  to  be  a  challenge 
because  pediatric  ambulatory  care  clinics  are  busy,  high-volume  settings  where 
hand  offs  are  common.  Pediatric  protocols  may  often  be  individualized  by 
clinicians,  and  children  thus  present  unique  opportunities  for  error.  Furthermore, 
each  clinic  may  be  organized  differently  based  on  its  location,  practice  type,  and 
size.  Additionally,  while  it  is  possible  to  leam  from  other  practices,  each  office 
practice  must  adapt  system  changes  meant  to  improve  quality  and  patient  safety  to 
meet  local  needs.  It  isn’t  enough  to  know  what  works  in  another  setting.  Each 
practice  must  recognize  how  to  assess  its  own  culture  and  design,  and  conduct 
effective  tests  of  change.  The  LEAP  study  is  an  important  initial  step  in  learning 
how  to  ensure  that  health  care  is  safer  for  children  and  families. 
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